Generation of GelSight Tactile Images for Sim2Real Learning

Most current works in Sim2Real learning for robotic manipulation tasks leverage camera vision that may be significantly occluded by robot hands during the manipulation. Tactile sensing offers complementary information to and can compensate loss caused occlusions. However, use of tactile is restricted research due no simulated sensors being available. To mitigate gap, we introduce a novel approach simulating GelSight sensor commonly used Gazebo simulator. Similar real sensor, produce high-resolution images from depth-maps captured optical reconstruct interaction between touched object an opaque soft membrane. It indirectly sense forces, geometry, texture other properties enables with sensing. Preliminary experimental results have shown could generate realistic outputs similar ones sensor. All materials this letter are available at https://danfergo.github.io/gelsight-simulation.